Performance of Multiplicative-weights-updates
نویسنده
چکیده
Let us remember the mechanics of Multiplicative-Weights-Updates: At every time t, the learner maintains a weight vector wt ≥ 0 over the experts. Given the weight vector, the probability distribution over the experts is computed as pt = wt wt·1 . The weights are initialized at w1 = 1 n · 1. (Multiplicative-weights-update step.) Given the loss vector at time t the weights are updated as follows wt+1(i) = wt(i) · uβ(lt(i)),∀i, where uβ : [0, 1]→ [0, 1] is an update function satisfying β ≤ uβ(x) ≤ 1− (1− β)x, ∀x ∈ [0, 1], for some β ∈ [0, 1]. The reader is free to chose whatever function uβ s/he wants. For example, one can use uβ(x) = β , for any β ∈ [0, 1]. In this case, wt+1(i) = wt(i) · βt = . . . = w1(i) · βt ≡ 1 nβ t.
منابع مشابه
Learning Linear Functions with Quadratic and Linear Multiplicative Updates
We analyze variations of multiplicative updates for learning linear functions online. These can be described as substituting exponentiation in the Exponentiated Gradient (EG) algorithm with quadratic and linear functions. Both kinds of updates substitute exponentiation with simpler operations and reduce dependence on the parameter that specifies the sum of the weights during learning. In partic...
متن کاملPath Kernels and Multiplicative Updates
Kernels are typically applied to linear algorithms whose weight vector is a linear combination of the feature vectors of the examples. On-line versions of these algorithms are sometimes called “additive updates” because they add a multiple of the last feature vector to the current weight vector. In this paper we have found a way to use special convolution kernels to efficiently implement “multi...
متن کاملOn Sex, Evolution, and the Multiplicative Weights Update Algorithm
We consider a recent innovative theory by Chastain et al. on the role of sex in evolution [10]. In short, the theory suggests that the evolutionary process of gene recombination implements the celebrated multiplicative weights updates algorithm (MWUA). They prove that the population dynamics induced by sexual reproduction can be precisely modeled by genes that use MWUA as their learning strateg...
متن کاملMultiplicative updates For Non-Negative Kernel SVM
We present multiplicative updates for solving hard and soft margin support vector machines (SVM) with non-negative kernels. They follow as a natural extension of the updates for non-negative matrix factorization. No additional parameter setting, such as choosing learning, rate is required. Experiments demonstrate rapid convergence to good classifiers. We analyze the rates of asymptotic converge...
متن کاملEfficient Multiplicative Updates for Support Vector Machines
The dual formulation of the support vector machine (SVM) objective function is an instance of a nonnegative quadratic programming problem. We reformulate the SVM objective function as a matrix factorization problem which establishes a connection with the regularized nonnegative matrix factorization (NMF) problem. This allows us to derive a novel multiplicative algorithm for solving hard and sof...
متن کامل